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LONG-TERM  GOALS 

To  develop  a  robust  automatic  classifier  with  a  high  probability  of  detection  and  a  low  false  alarm  rate 
that  can  classify  vocalizations  from  a  variety  of  cetacean  species  in  diverse  ocean  environments. 

OBJECTIVES 

In  previous  work  as  part  of  ONR  grant  N0001412 10139  a  unique  automatic  classifier  developed  by  the 
PI  that  uses  perceptual  signal  features  -features  similar  to  those  employed  by  the  human  auditory 
system-  was  employed  to  successfully  classify  anthropogenic  transients,  and  vocalizations  from  five 
cetacean  species.  Although  this  is  a  significant  achievement,  successful  implementation  of  this  (or 
any)  classifier  requires  that  it  be  temporally  and  spatially  robust.  The  primary  goal  will  be  to  address 
the  question:  “Will  it  work  on  vocalization  data  from  these  species  collected  under  different 
environmental  conditions?”  To  examine  this,  discriminant  analysis  will  be  used  to  rank  the  aural 
features  in  terms  of  their  ability  to  separate  the  vocalizations  between  species.  Then,  the  more  highly 
ranked  features  will  be  tested  for  robustness.  This  will  be  done  by  performing  a  propagation 
experiment  using  cetacean  vocalizations  and  synthetically  generated  calls  as  source  signals,  and  testing 
the  received  signals  with  the  classifier.  The  measurements  will  be  complemented  by  comparing 
experimental  results  to  propagation  model  results  with  the  goal  of  generalizing  the  results  to  other 
ocean  environments. 

APPROACH 

The  research  is  part  of  a  PhD  program  undertaken  by  Ms.  Carolyn  Binder  under  the  supervision  of  Dr. 
Paul  C.  Hines.  The  postgraduate  program  is  being  conducted  collaboratively  in  the  Oceanography  and 
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Electrical  Engineering  departments  at  Dalhousie  University  where  Dr.  Hines  holds  adjunct  professor 
and  research  posts,  and  at  Defence  R&D  Canada-Atlantic  where  Ms.  Binder  is  a  researcher. 

Passive  acoustic  monitoring  (PAM)  is  widely  in  use  to  study  marine  mammals;  since  marine  mammals 
can  be  found  in  all  ocean  basins,  their  habitats  cover  diverse  underwater  environments.  It  is  well 
known  that  acoustic  propagation  can  vary  substantially  between  environments  which  can  result  in 
distortion  of  acoustic  signals  [1-3].  This  in  turn,  can  lead  to  environment-dependent  time-frequency 
characteristics  of  a  received  vocalization.  The  resulting  distortion  of  vocalizations  may  impact  the 
accuracy  of  PAM  systems.  Thus,  to  develop  a  classification  system  capable  of  operating  in  many 
environments  one  must  understand  the  role  of  propagation  on  the  classifier. 

A  prototype  aural  classifier  developed  at  Defence  Research  and  Development  Canada  has  successfully 
been  used  for  inter-species  discrimination  of  cetaceans  [4],  The  aural  classifier  is  an  effective  PAM 
tool  because  it  employs  perceptual  signal  features,  which  model  features  used  by  the  human  auditory 
system  [5],  The  proposed  research  aims  to  examine  the  robustness  of  this  classifier,  and  the  perceptual 
features  it  uses,  to  environmental  conditions.  To  accomplish  this,  propagation  experiments  were 
conducted  by  transmitting  a  set  of  real  and  synthetic  vocalizations  from  bowhead  and  humpback 
whales,  and  measuring  the  received  signals  at  a  variety  of  ranges  [6].  The  transmitted  and  received 
signals  will  be  tested  using  the  aural  classifier  to  identify  any  performance  degradation  due  to 
propagation.  The  measurements  will  be  complemented  by  propagation-model  results  using 
environmental  inputs  measured  during  the  experiments.  The  model  results  will  provide  physical 
insight  into  what  propagation  effects  have  the  greatest  impact  on  the  classifier’s  robustness,  and  aid  in 
generalizing  the  experimental  results  to  other  ocean  environments.  It  also  enables  one  to  put  bounds 
on  realistic  within-environment  variability.  It  is  worthy  of  mention  that  there  is  no  study  published  in 
the  literature  that  systematically  analyzes  the  impacts  of  propagation  on  an  automated  classifier,  using 
both  underwater  propagation  experiments  and  complementary  modeling. 

If  propagation  does  impact  some  features,  then  the  next  step  is  to  rank  the  features  in  order  of  their 
sensitivity  to  propagation-related  effects.  Features  which  are  especially  sensitive  to  the  acoustic 
environment  might  simply  be  removed  from  the  aural  classifier.  Alternatively,  it  may  be  found  that 
many  of  the  perceptual  features  are  environment-sensitive  and  therefore  it  is  unreasonable  to  exclude 
all  of  them.  In  this  case,  a  strategy  may  instead  be  developed  to  generate  training  sets  for  the  classifier 
that  take  propagation-related  signal  distortion  into  account;  this  could  be  done  either  by  acoustically 
distorting  the  training  data  by  propagating  them  through  a  modeled  environment  or  by  including 
vocalizations  in  the  training  set  that  came  from  a  variety  of  propagation  ranges  as  done  in  [7,  8], 

WORK  COMPLETED 

The  focus  of  the  effort  during  FY  2015  has  been  two-fold: 

1 .  Continuing  detailed  analysis  of  the  data  collected  during  the  propagation  experiments  to  test  the 
robustness  of  the  aural  classification  feature  set.  Details  of  the  experiment  are  reported  in  [6] 
and  are  not  repeated  here  due  to  space  limitations,  but  a  schematic  of  the  experimental 
geometry  is  shown  in 

2.  Figure  1. 

3.  Exploring  the  relative  impacts  of  signal -to-noise  ratio  (SNR)  and  multipath  propagation  effects 
on  the  performance  of  the  classifier  using  a  pulse  propagation  model. 
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Experimental  Data  Analysis :  The  acoustic  data  that  were  collected  during  the  experiments  were 
processed.  The  processing  stream  is  as  follows:  The  signals  are  identified  in  the  recordings  made  by 
each  hydrophone  using  a  frequency  band-limited  energy  detector  [9].  These  detections  are  then 
compared  to  the  known  time  that  signals  were  transmitted  to  remove  false  detections.  After  each  signal 
has  been  detected,  a  four  second  segment  of  the  signal  is  extracted  with  the  detection  located 
approximately  in  the  centre  of  the  segment.  Each  extracted  detection  is  saved  to  a  WAV  file,  and  high- 
pass  filtered  to  remove  the  DC-offset  applied  by  the  recording  equipment  and  any  low-frequency 
noise.  At  this  point,  the  received  signals  are  able  to  be  input  to  the  aural  classification  algorithm. 
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Figure  1: :  Representation  of  the  experimental  setup.  The  ship  first  deployed  two  hydrophone 
moorings,  moved  to  the  first  location  and  transmitted  the  set  of  signals,  then  moved  further  away 
from  the  recorders  and  retransmitted  the  signals.  R1  and  R2  represent  the  horizontal  range  the 
signals  propagated  from  the  source  to  the  midpoint  between  the  moorings. 

The  58  perceptual  features  are  calculated  for  each  signal  transmitted  during  the  experiments  and  the 
signals  received  on  each  of  the  deployed  hydrophones.  An  initial  assessment  of  the  classifier 
performance  is  accomplished  by  training  the  classifier  on  data  recorded  by  a  monitor  hydrophone  that 
was  deployed  from  QUEST  during  the  experiments,  and  testing  on  data  transmitted  through  the  water. 
Classification  tasks  are  divided  so  that  the  real  and  synthetic  whale  calls  will  be  considered  separately. 

The  classifier  is  trained  using  all  non-redudant  features  (see  [5]  for  explaination  of  how  highly- 
correlated,  or  redundant,  features  are  identified)  as  determined  from  the  signals  in  the  training  set. 
Examining  and  comparing  the  decision  regions  for  signals  transmitted  over  each  of  the  ranges  will 
allow  qualitative  analysis  of  the  classifier’s  robustness.  Quantitative  analysis  will  consist  of  comparing 
the  accuracies  and  area  under  the  ROC  curve  AUCs,  as  well  as  the  class  means  and  variances.  If 
propagation  affects  the  perceptual  features  used  for  classification,  then  one  would  expect  that  the  class 
means  of  the  propagated  signals  in  the  training  and  testing  sets  would  be  significantly  different,  and/or 
there  would  be  a  large  change  in  the  variance  of  the  classes.  This  will  likely  result  in  a  decrease  in  the 
AUC  and/or  the  classification  accuracy.  This  type  of  analysis  is  repeated  allowing  the  classifier- 
algorithm  freedom  to  select  the  best  perceptual  features  to  train  the  classifier. 

Performance  Dependence  on  SNR  vs.  Multipath  Propagation:  Preliminary  results  from  the 
experimental  analysis  (Figure  2)  indicate  that  there  is  a  trend  of  decreasing  classifier  performance  with 
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increasing  transmission  range.  A  similar  pattern  was  noted  in  the  SNR-dependence  investigation  of 
Murphy  and  Hines  [7],  suggesting  that  SNR  may,  at  least  in  part,  be  driving  the  decrease  in  classifier 
performance.  Mouy  et  al.  [10]  also  noted  that  false  negative  rates  increased  as  SNR  decreased.  Thus,  it 
is  expected  that  there  is  a  range-dependent  component  of  the  aural  classifier  performance  that  is  due  to 
decreasing  SNR  as  the  range  increases.  The  question  thus  becomes,  what  are  the  relative  contributions 
of  lowering  SNR  and  increasing  multipath  signal  distortion  to  reduced  classifier  performance? 

Pulse  propagation  modelling  is  used  to  address  this  question.  DRDC’s  WATTCH  (WAveform 
Transmission  Through  a  CHannel)  [11,  12]  model  is  used  to  simulate  the  expected  time  series  received 
at  a  set  of  hydrophones  for  each  sourch  signal  based  on  the  user-defined  environment  inputs.  The 
propagatation  modelling  component  is  done  using  the  active  version  of  Bellhop.  The  WATTCH 
program  was  chosen  because  it  allows  for  range -dependent  environments,  is  efficient,  and  accuracte 
for  the  frequencies  and  spatial  scales  of  the  experiment.  To  examine  the  relative  importance  of  SNR 
and  propagation  on  classifier  performance,  a  simulation  study  was  developed  that  examined  three 
separate  cases: 

Case  1:  Add  noise  to  the  original  signals  (i.e.,  signals  which  were  not  transmitted  during 

experiment  or  simulation).  Snippets  of  noise  recorded  during  the  sea  trial  are  added  to  the 
signals  to  match  the  SNR  of  signals  that  were  recorded  during  the  experiments  for  ranges 
equal  to  R=  {0.07,  1,5,  10,  20}  km. 

Case  2:  Use  the  WATTCH  model  to  simulate  signals  propagated  over  range  R. 

Case  3:  Add  noise  to  simulated  signals  from  case  2,  to  match  the  measured  SNR  at  each  value  of  R. 
This  should  provide  the  most  realistic  results,  and  match  experimental  results. 


Classification  performance  for  each  of  these  cases  is  compared  as  a  function  of  propagtion  range. 

RESULTS 

Experimental  classifier  performance  for  the  synthetic  whale  calls  is  summarized  in  Figure  2.  The 
classifier  was  trained  with  data  from  a  monitor  hydrophone  deployed  from  QUEST,  separated  from  the 
acoustic  source  by  approximately  70  m.  The  data  transmitted  over  longer  ranges  is  then  run  throught 
the  trained  classifier.  Each  line  on  the  plot  shows  the  performance  determined  from  data  recorded  on  a 
different  hydrophone.  Results  are  also  separated  by  day.  There  is  a  clear  trend  for  the  classifier 
performance  to  decrease  with  increasing  range.  This  is  at  least  partially  due  to  classifier  performance 
decreasing  with  decreasing  SNR,  a  trend  that  is  obvious  in  the  bottom  panel  of  Figure  2.  The  question 
remains  -  what  is  the  relative  importance  of  SNR  and  multipath  propagation  effects  to  this  range- 
dependent  decrease  in  performance? 

A  modelling  study  was  performed  to  examine  this  question  by  comparing  classification  performance  as 
a  function  of  range  for  the  three  cases  discussed  above.  Results  are  shown  in 

Figure  3.  As  expected,  the  performance  decreased  as  SNR  decreased  with  increasing  range,  similar  to 
what  was  observed  in  the  experimental  data.  A  suprising  result  here  is  that  the  performance  remained 
perfect  for  the  second  case.  Even  though  multipath  propagation  increasingly  distorted  the  waveform 
with  range,  there  was  no  change  in  the  performance;  however,  when  noise  was  added  to  these  signals 
the  performance  decreased  below  that  of  the  noise  only  case.  This  suggests  that  with  sufficient  SNR, 
multipath  propagation  has  minimal  impact  on  classifier  performance;  lowering  the  SNR  however,  not 
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only  decreases  performance,  it  amplifies  the  multipath-dependent  degradation.  Further  work  needs  to 
be  done  to  determine  if  there  are  (realistic)  environments  in  which  propagation  effects  drive 
performance  and  whether  surface  roughness  needs  to  be  included  to  to  obtain  sufficient  fidelity  in  the 
model. 


Signal-to-Noise  Ratio  (dB) 


Figure  2:  Experimental  classifier  performance  for  bowhead  and  humpback  synthetic  calls  as  a 
function  of  transmission  range  (top)  and  SNR  (bottom).  The  classifier  was  trained  with  data  from  a 
hydrophone  deployed  from  QUEST  and  then  validated  on  data  from  each  of  the  recorders.  The 
black  circle  in  the  plots  represents  the  training  results. 
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Transmission  Range  (km) 

Figure  3:  Relative  impacts  on  classifier  performance  of  SNR  and  signal  distortion  due  to  multipath 
propagation  as  a  function  of  transmission  range.  The  classifier  was  trained  with  data  from  the  70  m 
range  and  applied  to  data  transmitted  over  the  longer  ranges. 

IMPACT/APPLICATIONS 

Detection  and  classification  of  cetaceans  has  become  critically  important  to  the  US  Navy  due  to  an 
ever  increasing  requirement  for  environmental  stewardship.  Passive  acoustics  continues  to  be  the  best 
method  to  carry  out  this  task  but  current  techniques  provide  only  a  partial  solution;  most  detectors  are 
either  too  general,  leading  to  unacceptably  high  false  alarm  rates,  or  are  too  specialized  (i.e.,  species- 
or  location-specific)  leading  to  many  missed  detections.  Furthermore,  future  military  platforms  will 
have  to  support  smaller  complements  and  deal  with  ever-increasing  data  throughput,  so  that 
automation  of  on-board  systems  is  essential.  In  addition,  the  technique  is  well  suited  to  autonomous 
systems  since  a  much  smaller  bandwidth  is  needed  to  transmit  a  classification  result  than  to  transmit 
raw  acoustic  data.  The  success  of  the  machine  classifier  in  discriminating  cetacean  vocalizations 
suggests  that  it  could  be  applied  to  other  passive  acoustic  classification  problems  which  currently 
employ  human  audition.  This  would  be  particularly  useful  if  expert  listeners  aren’t  available  -such  as 
diagnosing  heart  murmurs  in  remote  communities  that  lack  a  cardiologist,  or  as  part  of  the  triage 
process  in  a  hospital  emergency  department.  Alternatively,  the  machine  classifier  is  ideally  suited 
when  the  sheer  volume  of  data  makes  human  audition  untenable  -  such  as  classifying  ocean  acoustic 
data  for  species  population  monitoring.  Finally,  developing  a  robust  classifier  for  passive  marine 
mammal  vocalizations  is  also  a  first  step  to  testing  the  algorithm  on  passive  transients  generated  by 
submarines  to  examine  its  potential  for  passive  detection  and  classification  of  submarines. 

RELATED  PROJECTS 

This  research  will  benefit  from  DRDC  Atlantic’s  Force  ASW  Program  in  which  DRDC’s  aural 
classification  algorithms  (including  the  marine  mammal  classification  algorithm)  is  being  integrated 
into  DRDC’s  System  Test  Bed  (STB).  The  STB  is  used  to  evaluate  sonar  algorithms  in  a  military 
context.  Two  of  the  insights  to  be  gained  are:  1 .  Does  the  aural  classifier  reduce  false  alarms  from 
marine  mammals,  thereby  reducing  operator  workload  and  enabling  greater  concentration  on  potential 
threats?  2.  Is  the  aural  classifier  easily  integrated  into  a  navy  platform? 
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